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□ Overview of GC algorithms 

□ Case study: Sapphire on-the-fly replication collector 

□ Abstractions and invariants are essential for 
comprehension 

□ Design pattern for phase transitions 
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GC BACKGROUND 
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Styles of tracing collection 


□Mark-sweep 
□Copying GC 
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Styles of tracing collection 


□Mark-sweep 

□ Copying GC [Cheney, CACM, 1970] 
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Tricolour abstraction 



not yet been reached by the GC. 



visited by the GC, 

but fields need to be scanned. 



visited by the GC, 

all fields have been scanned. 
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Parallelism 


□Stop the world (STW) 
□Parallel GC 
□Incremental GC 
□Mostly concurrent GC 
□On-the-fly (OTF) GC 
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Parallelism 


□Stop the world (STW) 

□ Parallel GC 
□Incremental GC 
□Mostly concurrent GC 
□On-the-fly (OTF) GC 
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Parallelism 


□Stop the world (STW) 
□Parallel GC 
□Incremental GC 
□Mostly concurrent GC 
□On-the-fly (OTF) GC 
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Parallelism 


□Stop the world (STW) 
□Parallel GC 
□Incremental GC 
□Mostly concurrent GC 
□On-the-fly (OTF) GC 
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Parallelism 


□Stop the world (STW) 
□Parallel GC 
□Incremental GC 
□Mostly concurrent GC 
□On-the-fly (OTF) GC 
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SAPPHIRE 
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Sapphire 


□ On-The-Fly concurrent replicating GC 

[Hudson & Moss, Concurrency Practice & Experience, 2003] 

□ OTF: never stop more than one thread at a time 

□ Copying: fast allocation, eliminate fragmentation 

□ Synchronisation: GC pays (mostly) 

□ New implementation in Jikes RVM uikesrvm.org] 

□ Research JVM, “easy" to replace components 

□ Largely written in Java 
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Phases 


Before 
Mark phase 
Copy phase 
Flip phase 
lAfter 


Header 

Data 



from-space 

to-space 


Header 

Data 
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Phases 


□ Before 
□Mark phase 
□Copy phase 

□ Flip phase 
□After 


Header 

Data 



Header 

Data 


from-space 


to-space 
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Phases 


□ Before 
□Mark phase 
□Copy phase 

□ Flip phase 
□After 


Header 

Data 


read from 
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Header 

Data 


from-space 


to-space 


'write to 
both spaces 


Header 

Data 


‘semantic’ 

copy 
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Phases 


□ Before 
□Mark phase 
□Copy phase 

□ Flip phase 
□After 


Header 

Data 


write to 
both spaces 
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Phases 


□ Before 


□Mark phase 


□Copy phase 
□ Flip phase 
□After 



from-space 


to-space 


Header 

Data 



Header 

Data 
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WHAT COULD POSSIBLY 
GO WRONG? 
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The lost object problem 


□ Mutator ‘hides’ a 
reachable object 
from GC 



X 
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The lost object problem 


□ Mutator ‘hides’ a 
transitively reachable 
object from GC 
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Tricolour abstraction to the rescue! 
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Wilson conditions rwason. iwmm 92 ] 


□ Unsafe collection of a reachable object if and only if 

1 . A mutator stores a pointer to a white object into a black 
object 

□ AND 

2. All paths from any grey objects to that white object are 
destroyed. 

□ Enforce invariants to prevent one or other of these 
conditions from arising.... 
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Tricolour invariants 


□ Strong invariant: 

No pointers from black 
to white objects. 

□ Weak invariant: 

All white objects pointed 
to by a black object 
are reachable 
from some grey object, 
either directly or through 
a chain of white objects 
(“grey protected”). 



I Prevent 
jCond. 1 



j Prevent 
!Cond. 2 
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Barriers to maintain invariants 


□ Maintain invariants with barriers on writes (or 
reads). 

□ A mutator action that communicates with the collector. 
Typically, some code added around a pointer read or 
write (e.g. putfield bytecode). 


□ Insertion (incremental update) write barrier. 

□ Deletion (snapshot at the beginning) write 
barrier. 
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Mutator colours [Pirinen, ISMM 98 ] 


□ Grey mutator: 

roots still to be traced, or need to be rescanned 

■♦may point to black, grey or white objects 


□ Black mutator: 

roots have been traced, will not be rescanned 

^(strong invariant) will never point to white objects 

^(weak invariant) may point to grey protected white 
objects 
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Mutator techniques 


□ Grey mutator: 

□ strong invariant — 
insertion write barrier 
[Steele, CACM75; 
Dijkstra, 1976, CACM78] 

□ Black mutator: 



weak invariant — 
deletion write barrier 

[Abraham & Patel, ICPP87; 
Yuasa, JSS90] 

strong invariant — 
read barrier [Baker, CACM78] 



white or grey 
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TERMINATION 
and INITIALISATION 
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Termination of trace 


□ Grey mutator 

□ Trace until no grey 
objects remain. 

// Mutators may hold grey 
or white references. 

□ Loop: 

Scan each thread, 
shading any white 
object found. 

If any grey objects 
exposed 

then trace grey objects. 

Else break. 


□ Black mutator 

□ Trace until no grey objects 
remain. 

Even with the weak tricolour 
invariant the mutator can 
contain only black references. 

- There are no grey objects. 

- So there are no white objects 
reachable from grey objects. 

- Because the mutator is black 
there is no need to rescan its 
roots. 
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Initialisation of trace 


□ Mostly concurrent collectors... 

□ Stop the world briefly to scan mutator thread stacks 

□ Can use insertion or deletion write barrier, e.g. 

black mutator / deletion barrier / allocate new objects black 

□ On-the-fly collectors 

□ Deletion barrier not sufficient since... 

□ Mixture of black (scanned) and white (unscanned) stacks 

□ Must use insertion and deletion barrier until all roots scanned 
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PHASE TRANSITIONS 
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Stop-the-world / Mostly concurrent 
collection 


□ Phase changes are synchronous 

□ Once the collector has entered a new phase B, 
no mutator observes a GC state S A corresponding 
to the previous phase A. 

□ At any time, all mutators have a coherent GC 
state — all observe S A or all observe S B . 


GC phase A 
Mutator phase A 
Mutator nhnse A 
Mutator phase A 




^^GCphas^B 

Mutator phase B 
Mutator phase B 
Mutator phase B 
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On-the-fly collection 


□ Phase changes are ragged. 

□ A mutator will not recognise that the phase has 
been changed until it reaches a GC-safe point. 


□ Consequences: 

1 . Collectors cannot start work until it has determined that all 
mutators have recognised that the phase has been 
changed. 

2. Different mutators may observe different GC states. 
Invariants required by different GC states may be 
incompatible! 
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Phase Transition Design Pattern 


Type I: “recognising phase change” 

Insert an intermediate GC phase I: 

GC simply handshakes with every mutator at their GC-safe 
points 


GC phase A 


GC phase 



GC phase B 


Type II: “incompatible invariants” 

Insert two GC intermediate phases I, and l 2 ; 

write barrier for an intermediate phase is typically more 

complex 
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TRANSACTIONAL SAPPHIRE 
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Intel’s Haswell 


□ Transactional Memory Extensions (TSX-NI) 

□ (Restricted) Transactional Memory 

□ ...with limited processor support 


□ XBEGIN ... XEND 

□ Up to ~ 1 6 KiB of read and writes 
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Complexities 


□ Setup of transaction is expensive (3x CAS) 

□ Fallback required if transaction fails 

□ Aborted transactions are expensive 
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Sapphire copying 


Mutator must 
write to both 
replicas 


Header 

Data 



1 

1 
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Header 

Data 


from-space 


to-space 


'write to 
both spaces 


Header 

Data 


‘semantic’ 

copy 


[Ritson et al, ISMM14] 
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CAS 


u = load (to-space) 
v = load (from-space) 
if (u != v) 

compare-and-swap (to-space, u, v) 
restart 

else 

break 


University of 

Kent 


Computing 


VMSS, May 2016 


41 





Unsafe 


v = load (from-space) 
store (to-space, v) 
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HTM 


XBEGIN 

vi = load (from-space) 
store (to-space, vi) 

V2 = load (from-space) 
store (to-space, V2) 


XEND 
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MHTM 


XBEGIN 
copy object 1 
copy object 2 


copy object n 
XEND 
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Planned MHTM 


scan and record object 1 .. n 


XBEGIN 
copy object 1 
copy object 2 
• • • 

copy object n 


Take as much work 
] as possible out of 

VH transaction 


XEND 
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STM 


vi = load (from-space) 
store (to-space, vi) 

V2 = load (from-space) 

| NB. Mutator not 

store (to-space, V2) 1 reading to-space 

• • • 

MFENCE 

verify to-space with from-space 
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Raw copying speed 


■ 






Copy speed v 
transaction size 
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Concurrent copying speed 
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BUT IS IT CORRECT? 
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Assertions + Sanity checking 


□ Assertions 

□ Assertions are the GC developer’s friend 

□ Use copiously 

□ Sanity checking 

□ Jikes RVM provides a sanity checker 

□ Use after a GC e.g. 

□ Is every reference valid? 

□ Or do some point into reclaimed space? 
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Testing 


□ Assertions and sanity checking are just a form of 
testing. 

□ Non-determinism. 

Concurrently scheduled threads. 

Collections at different times. 

Relaxed memory. 

□ How many invocations to give confidence of 
correctness? 

10? 100? 500? 1,000? 

□ What’s the measure of “correct”? 

Unmodified Jikes RVMwill nof succeed 1000/1 000 fimes. 

My version fails no more often than the unmodified version?? 
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MODEL CHECKING 
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Bounded Model Checking 


□ Verification technique for asynchronous process 
systems 

□ Verify some property such as an invariant 

□ Model checker will... 

□ visit all possible states reachable from the initial state 

□ check whether a given property holds in every state 

□ =>Model must be small — time and space 
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SPIN 


□ Specification in the Promela language 

□ Sequential processes 

□ Asynchronous communication via channels 

□ Shared variables 

□ Properties = Linear Temporal Logic assertions 
injected into the model 

[Gerard Holzmann, The SPIN A/lode/ Checker: Primer and Reference 
Manual . Addison-Wesley, 2004] 
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Model 


□ Collector thread process 

□ Mutator thread process 

□ X86 relaxed memory process 

□ CPU stores inserted into its local FIFO store buffer 

□ Store-load forwarding: core can see any store sin its store 
buffer 

□ Only the core that issued *a = v can read the latest value v 

□ Assume cache coherency 


University of 

Kent 


Computing 


VMSS, May 2016 


56 





CPU model 


University of 
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Channels between a thread process and the 
memory process 

#define MUTATOR_WRITE (a, v) 
atomic { 

mutator_queue ! a,v; 


v 


Send <a,v> to 
mutator_queue channel 


mutator_local_memory[a] = v; 
mutator_queue_count[a]++ 


} 


c 

Computing 


#define MUTATOR_READ(a, v) 
atomic { 
if 

:: mutator_queue_count[a] == 0 
-> v = shared [a] 

:: else 

-> v = mutator_local_memory[a] 
fi 

\ VMSS, May 201 6 



Store-load 

forwarding 
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#define COMMIT_WRITE (q, count) 
(len(q) > 0) -> q ? a,v 

-> shared[a] = v; count[a]-- 



Commit write to 
shared memory 


active proctype memory() { 
do 

:: atomic{COMMIT_WRITE(mutator_queue, mutator_queue_count)} 

:: atomic{COMMrr_WRITE(collector_queue, collector_queue_count)} 
od 

> 
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Scenario 


□ Single object with a single non-reference field. 

□ Each semi-space contains one object. 
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Mutator 


proctype mutatorf) { 
byte x = 0, y; 
do 

:: true -> 
x = 1 - x; 
MUTATOR_WRITE(fromSpace, 
MUTATOR_WRITE(toSpace, x); 
:: true -> 
if 



:: Iflipped -> MUTATOR_READ(fromSpace, y) 
:: else -> MUTATOR_READ(toSpace, y) 


assert(x == y); 


od; 



Check that the 
mutator reads 
what it wrote 
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proctype collectorf) { 
byte rl , r2, tmp; 
do 

"true -> 

COLLECTOR_READ(fromSpace, rl ); 
COLLECTOR_READ(toSpace, r2); 
if 


::( r1 == r2) -> br eak 

::else -> atomic { /* CAS */ 

COLLECTOR_READ(to$pace, tmp); 
if 

::(r2 == tmp) -> COLLECTOR_WRITE(toSpace, rl ) 
::else -> break 



COLLECTOR_MFENCE — - 

Flush the 
store buffer 

v J 


COLLECTOR_MFENCE; 
flipped = true; 
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Collector: CAS 



proctype collector!) { 
byte x, y; 
do 

::true -> 

COLLECTOR_READ(fromSpace, x); 
COLLECTOR_WRITE(toSpace, x); 
COLLECTOR MFENCE: 


COLLECTOR_READ(fromSpace, y); 
COLLECTOR_READ(toSpace, x); 
if 

::(x == y) -> break 
::else -> skip 
fi 


verify 


od; 

COLLECTOR_MFENCE; 
flipped = true; 


} 



Handshake and 
change phase 
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Collector: STM 



Hashcode() 


□ hashCode() must consistently return the same integer... 

□ Address-based hashing: obj.hashcode() = address of obj 


□ Copying GC moves objects! 


0x12345600 

0x12345604 


Header 

HASHED 

Data 
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Hashing 
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Java Reference types 


□ Strong - usual references. 

□ Soft - used for caches that the GC can reclaim. 

□ Weak - used for canonicalised mappings that 
do not prevent the GC from reclaiming their 
keys or values. 

□ Phantom - used for scheduling pre-mortem 
cleanup actions more flexibly than finalisers. 
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Reference. get() 


Reference 

r — 

\ 

object 

k 

> 



Normal 

object 


□ SoftReference.get() returns a strong reference to its target 
or null if the GC has cleared the referent. 

□ WeakReference.get() returns a strong reference to its 
target or null if the GC has cleared the referent. 

□ PhantomReference.get() always returns null. 

□ Effectively, get() may make an object that was safe to 
reclaim unsafe fo reclaim! 
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Reachable objects 


□ Reachability is defined informally in the 
java.lang.ref package. 

□ Strongly: can be reached without traversing any 
reference objects. 

□ Softly: not strongly reachable but can be reached by 
traversing a soft reference. 

□ Weakly: neither strongly nor softly reachable but can be 
reached by traversing a weak reference. 

□ Phantom: neither strongly, softly nor weakly reachable, 
it has been finalised, and some phantom reference 
refers to it. 
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Reachable objects 


□ Reachal o^GC will not ^qnally in the 

java.Ia nQy reclai m' ^ 

□ Strongly: can be reachedwithout traversing any 
reference objects. 

□ Softly: not strongly reachable but can be reached by 
traversing a soft reference. 

□ Weakly: neither strongly nor softly reachable but can be 
reached by traversing a weak reference. 

□ Phanfom: neither strongly, softly nor weakly reachable, 
it has been finalised, and some phantom reference 
refers to it. 
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Reachable objects 


Reacha 
java. Ian 

3 Strongly: 
referenc 



GC will not 


aim 


GC may 
reclaim 



ally in the 


,t traversing any 


Softly: not strongly reachable but can be reached by 
traversing a soft reference. 

Weakly: neither strongly nor softly reachable but can be 
reached by traversing a weak reference. 


□ Phantom: neither strongly, softly nor weakly reachable, 
it has been finalised, and some phantom reference 
refers to it. 
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Reachable objects 


tally in the 

t traversing any 


can be reached by 


□ Weakly: neither strongly nor softly reachable but can be 
reached by traversing a weak reference. 

□ Phantom: neither strongly, softly nor weakly reachable, 
it has been finalised, and some phantom reference 
refers to it. 


Reacha 
java. Ian 

3 Strongly: 
referenc 

3 Softly: n 
traversin 
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Atomicity requirement 



□ java.lang.ref.Reference.get() returns a strong 
reference 

=> race between collector and mutator. 


□ If GC decides to reclaim a softly reachable 
target O it must clear atomically 

□ all soft references to O (e.g. reference from A), and 
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“Specification” 


□ « An object is softly reachable if it is not 
strongly reachable but can be reached by 
traversing a soft reference. » 

□ The java. lang. ref definitions are vague, e.g. 
they do not 

□ specify how many soft references we can traverse 
and when. 

□ require that there be no weak or phantom references 
in the chain. 


Ditto for other reference types... 
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Formalisation 


□ Relations, R £ P((Objects u Roots) x Objects) 

□ e.g. StrR, SoftR, WeakR, PhantR 

□ Transitive closure, 

TC(X, R) = { o £ Objects I 3 x e X . x R* o } 

□ e.g. StrongReachable = TC(Roots, StrR) 
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Correct specification 


□ SoftReachable = TC (Roots, StrR U SoftR)-StrongReachable 

□ WeakReachable = TC (Roots, StrR U SoftR U WeakR) 

- StrongReachable - SoftReachable 

□ PhantomReachable = 

(TC(Roots, StrR U SoftR U WeakR U PhantR) D Finalised ) 
- StrongReachable - SoftReachable - WeakReachable 
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Clearing references 


□ If the GC decides to clear a soft (weak) reference to 
a softly (weakly) reachable object o, it must also 
clear all soft (weak) references to the sets 

□ softToCleario ) = { w e SoftReachable I w StrR* o } 

□ weakToCleario) = { w e WeakReachable I w ( StrR u SoftR)* o } 

□ Note 

phantomToClear(o) = {} 
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Reference type usage 



CD 

CD 

O 

GO 

o 

u 


T = 4480ms 

2 

T = 3790ms 

SUnflOW2009 

1 

xalan2009 

dlL 

0* 



Normalised execution time 
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Options for implementation 


□ To process reference types, we could 

□ Stop the world, or 

□ Block any mutator that calls get() [Domani et al, 2000] or 

□ Process objects on-the-fly, never blocking a mutator 
other than to scan its roots 


[Ugawa et al, ISMM1 4] 
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GC State 



■» by collector 


£ by collector (atomic) 

> by mutator (atomic) 


start tracing 


NORMAL) starttracin< 3 / TRACING) n0traCir9W0rk ^ CLEARING) 
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Insertion-barrier code 


collection () { 

insertionBarrier ON; 
transitiveClosureFromRoot(); 
while (true) { /* try to terminate */ 

refState TRACING; 
handshakef); 

transitiveClosureNoRootScanf); 

scanRoot(); 

if (workQueue.emptyf) && 

CAS (refState, TRACING, CLEANING)) 
break; 

} 

insertionBarrier <— OFF; 

clearReference(); 

refState NORMAL; 

handshakef); } 

reclaim!); } 

} 

University of 

Kent 


get() { 

while (true) { 
switch (refState) { 
case NORMAL: case REPEAT: 
return referent; 
case TRACING: 


COLOR(referent)^WHITE) 


if(referent=null | 
return referent; 

CASfrefState, TRACING, REPEAT); 
break; /* retry */ 
case CLEANING: 
if(referent=null | 


COLOR (referent)^WH ITE) 


return referent; 
return null; 
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Insertion-barrier code 


collection () { 

insertionBarrier ON; 
transitiveClosureFromRoot(); 
while (true) { /* try to terminate */ 

refState TRACING; 
handshakef); 

transitiveClosureNoRootScanf); 

scanRoot(); 

if (workQueue.emptyf) && 

CAS (refState, TRACING, CLEANING)) 
break; 

> 


get() { 

while (true) { 
switch (refState) { 
case NORMAL: case REPEAT: 
return referent; 
case TRACING: 

if(referent=null | | COLOR(referent)^WHITE) 
return referent; 

CAS(refState, TRACING, REPEAT); 
break; /* retry */ 
case CLEANING: 

if(referent=null I | COLOR(referent)^WHITE) 


insertionBarrier <— OFF; 
clearReference(); 
refState NORMAL; 
handshakef); 
reclaim!); 

} 


No guarantee 
of termination 
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collection () { 

insertion Barrier <— ON; 
transitiveClosureFromRoot() ; 
deletion Barrier <— ON; 
handshake (); 
scanRoot(); 
insertion Barrier <— OFF; 
while(true) { 
refState <— TRACING; 
handshake/); 

transitiveClosureNoRootScan(); 
if(workQueue.empty() && 

CAS (refState, TRACING, 
CLEANING)) 
break; 

} 

deletion Barrier <— OFF; 
clearReferences(); 
refState <— NORMAL; 
handshake/); 
reclaim (); 


get() { 

while(true) { 
switch (refState ) { 
case NORMAL: 

return referent; 
case REPEAT: 

if (referent=null | | COLOR (referent)^WHITE) 
return referent; 

COLOR (referent) <— GREY; 

return referent; 
case TRACING: 

if (referent=null | | COLOR (referent)^WHITE) 
return referent; 

if (CAS (refState, TRACING, REPEAT)) { 
COLOR (referent) <— GREY; 
return referent; 

} 

break; /* retry */ 
case CLEANING: 

if (referent=null | | COLOR (referent)^WHITE) 
return referent; 

return null; 
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Deletion-barrier code 



collection () { 

insertion Barrier <— ON; 
transitiveClosureFromRoot() ; 
deletion Barrier <— ON; 
handshake (); 
scanRoot(); 
insertion Barrier <— OFF; 
while(true) { 
refState <— TRACING; 
handshake/); 

transitiveClosureNoRootScan(); 
if(workQueue.empty() && 

CAS (refState, TRACING, 
CLEANING)) 


get() { 

while(true) { 
switch (refState ) { 
case NORMAL: 
return referent; 
case REPEAT: 

if (referent=null | | COLOR (referent)^WHITE) 
return referent; 

COLOR (referent) <— GREY; 
return referent; 
case TRACING: 

if (referent=null | | COLOR (referent)^WHITE) 
return referent; 

if (CAS (refState, TRACING, REPEAT)) { 
COLOR (referent) <— GREY; 


} 


break; 

} 

deletion Barrier <— OFF; 
clearReferences(); 
refState <— NORMAL; 
handshake/); 
reclaim (); 


j^iurn referent; .. 

Termination 

?(roferent)#WHITE) 

guaranteed 

I 
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Deletion-barrier code 



REFERENCE TYPE PROCESSING 
EVALUATION 
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Pause time distribution 
blocking methods 
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Execution times 
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Termination without STW 


□ While GC is TRACING, mutator.get() may make a weakly 
reachable white object strongly reachable 

□ Insertion barrier 

□ repeatedly scan roots and trace until work queue is empty 

□ If TRACING, get() white referent sets state atomically to 
REPEAT and retries get() 

□ Deletion barrier 

□ If TRACING, get() white referent sets state atomically to 
REPEAT and retries 

□ Shades white referent grey in both REPEAT and TRACING 
states 
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Model checking termination 


□ Properties 

□ PI : Safety A mutator will never see a reclaimed object. 

□ P2: Consistency Once any get() method called on a 
reference object returns null, a mutator will never see 
the referent of that object. 

□ P3: Termination GC eventually terminates. 

□ Model checking 

□ PI, P2 hold 

□ P3 does not hold for the insertion barrier solution 

□ P3 does hold for the deletion barrier solution 
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Model checking: evaluation 


□ Practical. 

□ Easy to construct and check models. 

□ Bounded model checking cannot give a 100% guarantee of 
correctness... 

□ ...but gives confidence. 

□ Concurrent garbage collection is complex and it is easy to 
overlook corner cases. 

□ Model checking offers reasonable confidence in an 
algorithm at a reasonable cost. 
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Punchline 


□ Formal specification of reference type 
behaviour. 

□ Model-checked implementation. 

□ OTF reference processing phases are longer in 
the worst case but, with deletion barriers, not by 
much. 

□ Overall execution time is not increased 
significantly by processing references OTF, and 
is often reduced. 
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Summary 


□ On-the-fly copying collection for a full JVM is 
extremely complex. 

□ Abstractions and invariants are the key to 
comprehension 

□ Model checking is a practical way to provide 
confidence in algorithms 
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Distribution of task times 
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Verification 


□ Automated Verification of Practical Garbage 
Collectors, Chris Hawblitzel and Erez Petrank. 
POPL09. 

□ A Certified Framework for Compiling and 
Executing Garbage-Collected Languages, 
Andrew McCreigh, Tim Chevalier and Andrew 
Tolmach. ICFP10. 

□ Relaxing Safely: Verified On-fhe-Fly Garbage 
Collection for x86-TSO, Peter Gammie, Tony 
Hosking and Kai Engelhardt. PLDI15. 
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